Skip to content

fix(fspy): unify shared memory on a sparse temp file across all platforms - #576

Open
wan9chi wants to merge 1 commit into
mainfrom
fspy-sparse-file-shm
Open

fix(fspy): unify shared memory on a sparse temp file across all platforms#576
wan9chi wants to merge 1 commit into
mainfrom
fspy-sparse-file-shm

Conversation

@wan9chi

@wan9chi wan9chi commented Jul 28, 2026

Copy link
Copy Markdown
Member

Motivation

Codex CLI's and Claude Code's default sandboxes deny the primitives fspy's Unix shared memory was built on. The macOS Seatbelt profile denies shm_open (#563), and both sandboxes block Unix domain sockets, which the Linux memfd broker depended on. Plain files in the temp directory work under both sandboxes; the IPC lock file lives there today.

This PR replaces all three platform backends with one file-backed implementation, mapped through memmap2 on every platform. The API has three types, each one platform concept:

  • ShmKeeper is the name. create returns it, it carries the identifier (the backing file's absolute path), and dropping it removes the file with remove_file.
  • ShmHandle is the opened file. create returns one, so the creator never looks its own file up by name, and open returns one to everybody else. map can be called more than once.
  • Mapping is the bytes. It keeps them alive until dropped and cannot affect the name.

Removal works on every platform because modern Windows deletes with POSIX semantics: the name goes away at once, existing handles keep working, and mapped views keep the data alive. CI probes on Windows Server confirmed both, with a live writable view and with an open share-delete handle. The docs reserve the right to fail the delete while a view is mapped, and Windows versions without POSIX delete do fail it, so the keeper falls back to reopening the file with FILE_FLAG_DELETE_ON_CLOSE and closing it. Unit tests pin the full removal semantics: name gone with a live mapping, name gone with an open handle, and the handle still mapping the same bytes afterwards.

Name removal is cleanup. The channel invalidates contents inside the shared bytes, so nothing depends on removal timing. Backing files sit directly in the system temp directory as vite-task-fspy-<uuid>.shm with mode 0o600; a shared subdirectory would belong to whichever user created it first and lock everyone else out. The identifier is resolved to an absolute path at creation, so a relative TMPDIR in the creating process cannot mislead an opener with a different working directory.

There is no broker, no global object name, no tokio requirement, and no hand-written mapping code: the Windows-specific parts shrink to the sparse-file FSCTL and the creation flags.

If the keeper's process is killed, the file stays behind: on Unix for the temp reaper, on Windows until a cleanup tool runs. It costs about as much disk as the run wrote into it.

Refs #563.

🤖 Generated with Claude Code

wan9chi commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -2.47%  [ -7.64% ..  +3.12%]  overhead   +47.04%
dynamic/access             change  +0.18%  [ -0.91% ..  +0.83%]  overhead   +54.54%
static/launch              change  +2.23%  [ -5.37% .. +10.35%]  overhead  +155.48%
static/access              change  -1.82%  [ -3.62% ..  -0.15%]  overhead  +829.10%

macos

dynamic/launch             change  +1.07%  [ -1.58% ..  +3.60%]  overhead  +219.65%
dynamic/access             change  -0.30%  [ -2.47% ..  +1.55%]  overhead   +20.62%

windows

dynamic/launch             change  -0.69%  [ -7.73% ..  +7.69%]  overhead   +30.47%
dynamic/access             change  +0.99%  [ -2.30% .. +41.56%]  overhead    +9.25%

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f029ee6c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/fspy_shm/src/file_backed.rs Outdated
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch 6 times, most recently from 5e527ad to f572f0d Compare July 30, 2026 02:14
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch from f572f0d to ca1932a Compare July 30, 2026 03:24
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread CHANGELOG.md Outdated
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch 3 times, most recently from 54dbb9b to 126f02b Compare July 30, 2026 03:51
…orms

Codex CLI's and Claude Code's default sandboxes deny the primitives fspy's
Unix shared memory was built on: the macOS Seatbelt profile denies
`shm_open` (`ipc-posix-shm-write-create`), and both sandboxes block
Unix-domain sockets, which the Linux memfd broker depended on. Plain files
in the temp directory are writable under both.

Replace all three backends with one file-backed implementation attached by
path: a sparse temporary file plus `memmap2`, with the file's absolute path
as the identifier. No broker, no tokio requirement, no global object names.

Lifetime semantics converge too: dropping the owner makes the name
disappear and later opens fail, while existing views stay usable — now on
Windows as well, where closing the delete-on-close handle applies the delete
disposition even while other processes hold views.

Refs #563.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch from 126f02b to c86913c Compare July 30, 2026 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant